2 research outputs found
Graphical Object-Centric Actor-Critic
There have recently been significant advances in the problem of unsupervised
object-centric representation learning and its application to downstream tasks.
The latest works support the argument that employing disentangled object
representations in image-based object-centric reinforcement learning tasks
facilitates policy learning. We propose a novel object-centric reinforcement
learning algorithm combining actor-critic and model-based approaches to utilize
these representations effectively. In our approach, we use a transformer
encoder to extract object representations and graph neural networks to
approximate the dynamics of an environment. The proposed method fills a
research gap in developing efficient object-centric world models for
reinforcement learning settings that can be used for environments with discrete
or continuous action spaces. Our algorithm performs better in a visually
complex 3D robotic environment and a 2D environment with compositional
structure than the state-of-the-art model-free actor-critic algorithm built
upon transformer architecture and the state-of-the-art monolithic model-based
algorithm
Gradual Optimization Learning for Conformational Energy Minimization
Molecular conformation optimization is crucial to computer-aided drug
discovery and materials design. Traditional energy minimization techniques rely
on iterative optimization methods that use molecular forces calculated by a
physical simulator (oracle) as anti-gradients. However, this is a
computationally expensive approach that requires many interactions with a
physical simulator. One way to accelerate this procedure is to replace the
physical simulator with a neural network. Despite recent progress in neural
networks for molecular conformation energy prediction, such models are prone to
distribution shift, leading to inaccurate energy minimization. We find that the
quality of energy minimization with neural networks can be improved by
providing optimization trajectories as additional training data. Still, it
takes around additional conformations to match the physical
simulator's optimization quality. In this work, we present the Gradual
Optimization Learning Framework (GOLF) for energy minimization with neural
networks that significantly reduces the required additional data. The framework
consists of an efficient data-collecting scheme and an external optimizer. The
external optimizer utilizes gradients from the energy prediction model to
generate optimization trajectories, and the data-collecting scheme selects
additional training data to be processed by the physical simulator. Our results
demonstrate that the neural network trained with GOLF performs on par with the
oracle on a benchmark of diverse drug-like molecules using x less
additional data.Comment: 17 pages, 5 figure